Polis is an open source wiki-survey platform for rapid, scalable, open ended feedback, in which participants submit short comments which are sent out semi-randomly to other participants to vote on (by clicking agree, disagree or pass). Polis uses statistical algorithms to find patterns of consensus and opinion groups.
This report looks at the data generated in an engagement run by Math & Democracy (pro-bono) in partnership with University of Kentucky, NPR and The American Assembly (part of Columbia University in February 2020. The poll asked residents of Louisville, Kentucky to respond to the question
What do you believe should change in Louisville to make it a better place to live, work and spend time?
Of the raw data collected, we have:
| Participants | Commenters | Comments | Votes | Agrees | Disagrees | Votes / participant (avg) | Groups |
|---|---|---|---|---|---|---|---|
| 1398 | 302 | 877 | 124787 | 66977 | 17451 | 89.26 | 2 |
After removing moderated out comments, and participants who voted on fewer than 7 comments, we have:
| Participants | Commenters | Comments | Votes | Agrees | Disagrees | Votes / participant (avg) |
|---|---|---|---|---|---|---|
| 1164 | 132 | 573 | 123139 | 65775 | 17237 | 105.78 |
Here we can see the distribution of these votes and comments over time as the conversation unfolded.
Next, we'll take a look at the variance in the data by plotting comments according to the number of agrees and disagrees. This data is plotted in a log plot due to the highly skewed nature of vote count distribution per comment. The grey line separates comments which were predominantly agreed with (bottom right) from those predominantly disagreed with (bottom left).
Note that comments with far more disagrees than agrees had overall much lower vote counts. This is a direct result of the comment routing architecture of Polis, which deprioritizes comments which most people disagree with.
We can take these votes and arrange them into a matrix, where rows correspond to participants and columns correspond to statements. This allows us to think of participants as having positions in a high dimensional space (dimensionality equal to the number of comments).